Elastic Net for Regression with Optimal Scaling Transformations

Regularized regression methods for linear regression have been developed the last few decades to overcome the flaws of ordinary least squares regression with regard to prediction accuracy. In this chapter, three of these methods (Ridge regression, the Lasso, and the Elastic Net) are incorporated into CATREG, an optimal scaling method for both linear and nonlinear transformation of variables in regression analysis. We show that the original CATREG algorithm provides a very simple and efficient way to compute the regression coefficients in the constrained models for Ridge gression, the Lasso, and the Elastic Net. The resulting procedures, subsumed under the term “regularized nonlinear regression” will be illustrated using the prostate cancer data, which have previously been analyzed in the regularization literature for linear regression. For model selection and the estimation of the prediction accuracy, we used the .632 bootstrap with the one-standard-error rule. We also show that the “CATREG-Lasso” with nominal transformations is equivalent to the